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DETAILED ACTION 



Response to Amendment 



1. 



Claims 1-23 are pending. 



2. 



Claims 1, 10-12, 21-23 have been amended. 



Response to Arguments 



3. Applicant's arguments filed 3/1 1/2009 have been fully considered but they are 
not persuasive. 

4. In response to applicant's argument that the references fail to show certain 
features of applicant's invention, it is noted that the features upon which applicant relies 
(i.e., "In contrast, claims 1, 10, 11 and 12 recite no such learning step of using training 
data nor do they require any training data at all." (Remarks, Page 9, U 4)) are not recited 
in the rejected claim(s). Although the claims are interpreted in light of the specification, 
limitations from the specification are not read into the claims. See In re Van Geuns, 988 
F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). 

5. Applicant further argues "An additional limitation among the limitations of 
independent claims 1 , 10, 11 and 1 2 that are neither disclosed nor suggested in the 
prior art of record are the requirements for "outputting an initial value of a model 
parameter which defines the [generated] probability model." (Remarks, Page 9, H 5) The 
Examiner disagrees, when a topic is determined, (column 5, lines 1-2) it is determined 
to be the topic for which the likelihood of the text is maximal (column 4, lines 55-67). If 
there has been a determination of a topic which maximizes the likelihood based on the 
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training procedure, (column 4, lines 5-25) the initial model parameters determined by 
the training process for the HMM have been output to the segmenter. The argument is 
not persuasive. 

6. Applicant further argues "The selection of a neutral topic does not teach or 
suggest any initial values for the parameters that define a probability model. Nowhere 
within the four corners of Kanevsky are there any teachings for initializing model 
parameters. (Remarks, Page 1 0, 1f 2) The Examiner disagrees for the same reasons as 
above. If there has been a determination of a topic which maximizes the likelihood 
based on the training procedure, (column 4, lines 5-25) the initial model parameters 
determined by the training process for the HMM have been output to the segmenter. 
The argument is not persuasive. 

7. In response to applicant's argument that the references fail to show certain 
features of applicant's invention, it is noted that the features upon which applicant relies 
(i.e., "estimating a model parameter corresponding to a text document as a processing 
target on the basis of the initial value of the model parameter.., and only the text 
document." (Remarks, Page 10, U 3)) are not recited in the rejected claim(s). Although 
the claims are interpreted in light of the specification, limitations from the specification 
are not read into the claims. See In re Van Geuns, 988 F.2d 1 181 , 26 USPQ2d 1057 
(Fed. Cir. 1993). 

8. Applicant further argues "Among the limitations of independent claims 21-23 that 
are neither disclosed nor suggested in the prior art of record are the requirements for 
"estimating a parameter of a probability model so that the probability of the text 
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document being output is maximized or locally maximized, wherein.., the model is fully 
defined by a model parameter set which is comprised of word output probabilities of 
words in the topics in the text document and topic transition probabilities."" (Remarks, 
Page 10, If 3) The Examiner disagrees. Kanevsky, column 4, lines 52-67, ...Find a 
candidate topic T, for which the likelihood of the text is maximal..., the likelihoods are for 
the text (word) which belongs to the topic (latent variable). Furthermore, the 
designation of a topic defines the probability of a word being output as it is linked to the 
maximal probability for the topic. Lastly, column 5, lines 3-65, ...some likelihood 
measure for "seeing" a given string of words... teaches the transition. As is described 
above, there is an initialization of model parameters which are used to determine the 
appropriate topics for a given string of words. 

9. In response to applicant's argument that the references fail to show certain 
features of applicant's invention, it is noted that the features upon which applicant relies 
(i.e., " Again, determining which topic among many learned from training data does not 
provide disclosure of estimating model parameters based on the initial values and only 
the text document, as required by the claims" (Remarks, Page 1 1 , If 1 )) are not recited 
in the rejected claim(s). Although the claims are interpreted in light of the specification, 
limitations from the specification are not read into the claims. See In re Van Geuns, 988 
F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). 

10. New art has been cited, (Blei, Utiyama, Koshinaka) in light of the amendments. 
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Claim Rejections - 35 USC § 102 

The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

1 1 . Claims 1-9 are rejected under 35 U.S.C. 102(b) as being anticipated by 
Koshinaka et al. (NPL document "An HMM-Based Text Segmentation Method Using 
Variational Bayes Approach and its Application to LVCSR for Broadcast News") 

As per claim 1 , Koshinaka teaches the method comprising: 

generating a probability model in which information indicating which word of a 
text document belongs to which topic is made to correspond to a latent variable and 
each word of the text document is made to correspond to an observable variable; 
(Page 486, Fig .1 , the observable variables are the transitions (ai), the latent (hidden) 
variables are (b).) 

outputting an initial value of a model parameter which defines the generated 
probability model; (Page, 486, Section 2.3, step 1 , prior distribution.) 

estimating a model parameter corresponding to a text document as a processing 
target on the basis of the output initial value of the model parameter and only the text 
document; (Page 486, Section 2.3, shows the estimation, Page 485, column 

1, ...our approach is unsupervised... teaches external training data.) 
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segmenting the text document as the processing target for each topic on the 
basis of the estimated model parameter. (Page 486, section 2.3) 

As per claim 2, claim 1 is incorporated and Koshinaka teaches: 

the step of generating a probability model comprises the step of generating a 
plurality of probability models, (Page 487, Section 2.4, ...multiple 

hypotheses...) 

the step of outputting an initial value of the model parameter comprises the step 
of outputting an initial value of a model parameter for each of the plurality of probability 
models, (Page, 486, Section 2.3, step 1 , prior distribution. The step is 

performed for all hypotheses, Page 487, Section 2.4) 

the step of estimating a model parameter comprises the step of estimating a 
model parameter for each of the plurality of probability models, and (Page 486, 

Section 2.3, shows the estimation which is performed for all hypotheses on N, Page 
487, Section 2.4) 

the method further comprises the step of selecting a probability model, from the 
plurality of probability models, which is used to perform processing in the step of 
segmenting the text document, on the basis of the plurality of estimated model 
parameters. (Page 487, Section 2.4, AIC, MDL) 

As per claim 3, claim 1 is incorporated and Koshinaka teaches: 

a probability model is a hidden Markov model. (Page 486, column 1 ) 
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As per claim 4, claim 3 is incorporated and Koshinaka teaches: 



the hidden Markov model has a unidirectional structure. 



(Fig. 1) 



As per claim 5, claim 3 is incorporated and Koshinaka teaches: 



the hidden Markov model is of a discrete output type. 



(Fig. 1, the markov 



operates in discrete time, also see Page 485, section 2.1 , ...a discrete HMM of N-state 
left-to-right architecture. . . ) 

As per claim 6, claim 1 is incorporated and Koshinaka teaches: 

the step of estimating a model parameter comprises the step of estimating a 
model parameter by using one of maximum likelihood estimation and maximum a 
posteriori estimation . (Page 486, Section 2.2) 

As per claim 7, claim 1 is incorporated and Koshinaka teaches: 

characterized in that the step of outputting an initial value of a model parameter 
comprises the step of hypothesizing a distribution using the model parameter as a 
probability variable, and outputting an initial value of a hyper-parameter defining the 
distribution, and (Page 486, Section 2.3, prior distribution, hyper-parameters) 

the step of estimating a model parameter comprises the step of estimating a 
hyper-parameter corresponding to a text document as a processing target on the basis 
of the output initial value of the hyper-parameter and the text document. 
(Page 486-487, Section 2.3, the hyper parameters are estimated and used to estimate 
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As per claim 8, claim 7 is incorporated and Koshinaka teaches: 

the step of estimating a hyper-parameter comprises the step of estimating a 
hyper-parameter by using Bayes estimation. (Page 486, Section 2.3, step 2) 

As per claim 9, claim 2 is incorporated and Koshinaka teaches: 

characterized in that the step of selecting a probability model comprises the step 
of selecting a probability model by using one of an Akaike's information criterion, a 
minimum description length criterion, and a Bayes posteriori probability. (Page 
487, Section 2.4) 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

12. Claims 1 0-23 are rejected under 35 U.S.C. 1 03(a) as being unpatentable over 
Koshinaka et al. (NPL document "An HMM-Based Text Segmentation Method Using 
Variational Bayes Approach and its Application to LVCSR for Broadcast News") 
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Claims 10 and 1 1 are rejected under the same principles for being the computer 
readable medium claims to the corresponding method claim 1 . Each of the stated 
corresponding claims have parallel limitations between the method and the recording 
medium of claims 10 and 1 1 . Koshinaka teaches an LVSCR system on page 485, 
Section 1 . It would have been obvious to someone of ordinary skill in the art at the time 
of the invention that an LVSCR system could have been hardware which requires a 
computer-readable medium to program the hardware. 

Claims 12-20 are rejected under the same principles for being the apparatus 
claims to the corresponding method claims 1-9. Each of the stated corresponding 
claims have parallel limitations between the method and the device and the hardware 
aspect of claims 12-20. Koshinaka teaches an LVSCR system on page 485, Section 1 . 
It would have been obvious to someone of ordinary skill in the art at the time of the 
invention that an LVSCR system could have been hardware. 

As per claim 21 , Koshinaka teaches: 

estimating a parameter of a probability model so that the probability of the text 
document being output is maximized or locally maximized, wherein the structure of the 
probability model is defined by latent variables representing which word of the text 
document belongs to one of plurality of topics, and the model is fully defined by a model 
parameter set which is comprised of word output probabilities of words in the topics in 
the text document and topic transition probabilities and (Page 486, section 2.2 
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describes an maximum likelihood estimation which estimates a parameter of a 
probability model such that the probability of the text document is maximized. The 
structure of the model is an HMM (Fig. 1) which has hidden states (latent variables) 
defining the topics. The model is fully defined by the parameter set including word 
output probabilities of words in the topics (hidden variables of the HMM) and the 
transitions between topics (See Fig. 1 , ai).) 

segmenting the text document for each topic by estimating the value of the latent 
variable for each word on the basis of the parameter of the probability model estimated 
above. (Section 2.3. Bayesian Segmentation) 

Claim 22 is rejected under the same principles for being the computer readable 
medium claims to the corresponding method claim 21. Each of the stated corresponding 
claims have parallel limitations between the method and the recording medium of claim 
22. Koshinaka teaches an LVSCR system on page 485, Section 1 . It would have been 
obvious to someone of ordinary skill in the art at the time of the invention that an 
LVSCR system could have been hardware which requires a computer-readable medium 
to program the hardware. 

Claim 23 is rejected under the same principles for being the apparatus claims to 
the corresponding method claims 21 . Each of the stated corresponding claims have 
parallel limitations between the method and the device and the hardware aspect of 
claims 23. Koshinaka teaches an LVSCR system on page 485, Section 1 . It would have 
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been obvious to someone of ordinary skill in the art at the time of the invention that an 
LVSCR system could have been hardware. 

13. Claims 1,3-6, 10-12, 14-17, and 21-23 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Blei et al. (NPL document "Topic Segmentation with an Aspect 
Hidden Markov Model") in view of Utiyama et al. (NPL document "A Statistical Model for 
Domain-Independent Text Segmentation") 

As per claim 1 , Blei teaches: 

generating a probability model in which information indicating which word of a 
text document belongs to which topic is made to correspond to a latent variable and 
each word of the text document is made to correspond to an observable variable; 
(Fig. 1, Page 344, column 2, ...The Viterbi algorithm [8], a dynamic programming 
technique, is used to find the most likely hidden sequence of topic states .. .given an 
observed sequence of word sets. . . ) 

outputting an initial value of a model parameter which defines the generated 
probability model; (Page 345, column 1 , ...In the E-step, we compute the 

posterior probability of the hidden variable given our current model. ..) 

segmenting the text document as the processing target for each topic on the 
basis of the estimated model parameter. (Section 3. HMM SEGMENTATION) 

Blei fails to completely teach, but Utiyama teaches: 
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estimating a model parameter corresponding to a text document as a processing 
target on the basis of the output initial value of the model parameter and only the text 
document; (Blei teaches an Expectation Maximization algorithm on page 345, 

column 1 which maximizes the parameters to fit the topics to the maximum value 
available by the training data. However, Blei fails to teach that the EM algorithm 
operates only on the text document itself (no training data). Utiyama teaches this on 
page 505, column 1 , ...Another major difference from their algorithm is that our 
algorithm does not require training data to estimate probabilities...) 

It would have been obvious to someone of ordinary skill in the art at the time of 
the invention to combine Utiyama with Blei to avoid using training data for text 
segmentation such that the method could be applied to text from any domain. (Utiyama, 
abstract) 

As per claim 3, claim 1 is incorporated and Blei teaches: 

a probability model is a hidden Markov model. (Abstract, ...We extend 

this idea by embedding Hofmann's aspect model for text [5] into the segmenting HMM 
to form an aspect HMM (AHMM)...) 

As per claim 4, claim 3 is incorporated and Blei teaches: 

the hidden Markov model has a unidirectional structure. (Fig. 1) 

As per claim 5, claim 3 is incorporated and Blei teaches: 
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the hidden Markov model is of a discrete output type. 



(Page 344, Fig. 1, 



the markov operates in discrete time) 

As per claim 6, claim 1 is incorporated and Blei teaches: 

the step of estimating a model parameter comprises the step of estimating a 
model parameter by using one of maximum likelihood estimation and maximum a 



(EM) algorithm...) 

Claims 10, 1 1 and 22 are rejected under the same principles for being the 
computer readable medium claims to the corresponding method claims 1 and 21 . Each 
of the stated corresponding claims have parallel limitations between the method and the 
recording medium of claims 10, 11, and 22. (Blei, Page 348, column 2) teaches a real 
time application of the method. It would have been obvious to someone of ordinary skill 
in the art at the time of the invention that a computer-based system would provide real- 
time functionality and a computer based system needs to be programmed by a 
computer readable recording medium in order to be functional according to the method. 

Claims 12-17 are rejected under the same principles for being the apparatus 
claims to the corresponding method claims 1, 3, 4, 5, and 6. Each of the stated 
corresponding claims have parallel limitations between the method and the device and 
the hardware aspect of claims 12-17 are taught by (Blei, Page 348, column 2) where it 



posteriori estimation. 



(Page 345, column 1, ...Expectation Maximization 
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would have been obvious to someone of ordinary skill in the art at the time of the 
invention that the SpeechBot system could have been a hardware computer system. 

As per claim 21 , Blei teaches: 

segmenting the text document for each topic by estimating the value of the latent 
variable for each word on the basis of the parameter of the probability model estimated 
above. (Section 3. HMM SEGMENTATION) 

Blei, however, fails to fully teach, but Utiyama teaches: 

estimating a parameter of a probability model so that the probability of the text 
document being output is maximized or locally maximized, wherein the structure of the 
probability model is defined by latent variables representing which word of the text 
document belongs to one of plurality of topics, and the model is fully defined by a model 
parameter set which is comprised of word output probabilities of words in the topics in 
the text document and topic transition probabilities and (Blei teaches an 
Expectation Maximization algorithm on page 345, column 1 which maximizes the 
parameters to fit the topics to the maximum value available by the training data. Page 
344, column 2, Fig. 1 shows the topics as the hidden states (latent variables). Blei fails 
to teach that the model parameters are fully defined in the text document. Utiyama 
teaches this on page 505, column 1 , ...Another major difference from their algorithm is 
that our algorithm does not require training data to estimate probabilities. ..) 
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It would have been obvious to someone of ordinary skill in the art at the time of 
the invention to combine Utiyama with Blei to avoid using training data for text 
segmentation such that the method could be applied to text from any domain. (Utiyama, 
abstract) 

Claim 23 is rejected under the same principles for being the apparatus claims to 
the corresponding method claim 21 . Each of the stated corresponding claims have 
parallel limitations between the method and the device and the hardware aspect of 
claim 23 is taught by (Blei, Page 348, column 2) where it would have been obvious to 
someone of ordinary skill in the art at the time of the invention that the SpeechBot 
system could have been a hardware computer system. 

14. Claims 2, 9, 1 3, and 20 are rejected under 35 U.S.C. 1 03(a) as being 
unpatentable over Blei et al. (NPL document "Topic Segmentation with an Aspect 
Hidden Markov Model") in view of Utiyama et al. (NPL document "A Statistical Model for 
Domain-Independent Text Segmentation") and further in view of Rabiner (NPL 
document "A tutorial on Hidden Markov Models and Selected Application in Speech 
Recognition"). 

As per claim 2, claim 1 is incorporated and Blei teaches: 

the generation of a probability model; (Fig. 1, Page 344, column 2, ...The Viterbi 
algorithm [8], a dynamic programming technique, is used to find the most likely hidden 
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sequence of topic states. . .given an observed sequence of word sets. ..) 

the step of outputting an initial value of a model parameter for the probability 
model; and outputting an initial value of a model parameter which defines the 

generated probability model; (Page 345, column 1 , ...In the E-step, we 

compute the posterior probability of the hidden variable given our current model. . . ) 

estimating a model parameter for the probability model. (Blei teaches 

an Expectation Maximization algorithm on page 345, column 1 which maximizes the 
parameters to fit the topics to the maximum value available by the training data.) 

Blei and Utiyama fail to teach, but Rabiner teaches: 

multiple probability models; (Rabiner, page 266, discloses multiple 

HMM models which are applicable to the HMM used in Blei. It would be obvious to 
someone of ordinary skill in the art that multiple models could be developed around the 
number of states or form of the HMM models to characterize them differently. Upon use, 
one will perform the best, so it would be obvious that one model would be chosen to be 
used for segmentation of the text document.) 

It would have been obvious to someone of ordinary skill in the art at the time of 
the invention to combine Rabiner with Blei and Utiyama to be able to choose a model 
that best represents the observed properties of the signal (Rabiner, Page 266) to use 
the observed properties to determine the best location for a topic change. 



As per claim 9, claim 2 is incorporated and Blei, Utiyama , and Rabiner fails to 
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specifically teach: 

the step of selecting a probability model comprises the step of selecting a 
probability model by using one of an Akaike's information criterion, a minimum 
description length criterion, and a Bayes posteriori probability 
(However, Akaike's information criterion is well known in the art for model selection. 
Since it is obvious to select a model, it would be obvious to someone of ordinary skill to 
use Akaike's information criterion to select a model to determine the best model for 
segmentation of the text.) 

Claims 13 and 20 are rejected under the same principles for being the apparatus 
claims to the corresponding method claims 2 and 9. Each of the stated corresponding 
claims have parallel limitations between the method and the device and the hardware 
aspect of claims 1 3 and 20 are taught by (Blei, Page 348, column 2) where it would 
have been obvious to someone of ordinary skill in the art at the time of the invention that 
the SpeechBot system could have been a hardware computer system. 

15. Claims 7-8, and 18-19 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Blei et al. (NPL document "Topic Segmentation with an Aspect 
Hidden Markov Model") in view of Utiyama et al. (NPL document "A Statistical Model for 
Domain-Independent Text Segmentation") and further in view of NPL document 
"Bayesian Adaptive Learning of the Parameters of Hidden Markov Model for Speech 
Recognition" hereinafter Huo). 
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As per claim 7, claim 1 is incorporated and Blei and Utiyama fails to teach, but Huo 
teaches: 

the step of outputting an initial value of a model parameter comprises the step of 
hypothesizing a distribution using the model parameter as a probability variable, and 
outputting an initial value of a hyper-parameter defining the distribution 
(Huo, page 335, ...we do not explicitly show the parameters of the prior PDF (often 
referred to as the hyperparameters) which are assigned values by the investigator since 
the values are assigned, and thus initialized...) 

the step of estimating a model parameter comprises the step of estimating a 
hyper-parameter corresponding to a text document as a processing target on the basis 
of the output initial value of the hyper-parameter and the text document 
(Huo, page 335, ...the important issue of prior density estimation is addressed and an 
empirical Bayes method to estimate the hyperparameters of prior density based on the 
moment estimate is proposed..., Furthermore, Huo, 339, teaches that equation 49 is 
used for updating the hyperparameters. Thus, they are based on the initial value and 
are estimated.) 

Huo and Blei are analogous art because Huo's paper concerns the training of the 
HMM model parameters that are used in Blei. It would be obvious to someone of 
ordinary skill in the art at the time of the invention to combine Huo with the Blei and 
Utiyama device because Huo provides algorithms that "are shown to be effective 
especially in the cases in which the training or adaptation data are limited" which would 
provide an improvement over previous algorithms. 
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As per claim 8, claim 7 is incorporated and Blei and Utiyama fails to teach, but Huo 
teaches: 

the step of estimating a hyper-parameter comprises the step of estimating a 
hyper-parameter by using Bayes estimation 

(Huo, page 335, ...the important issue of prior density estimation is addressed 
and an empirical Bayes method to estimate the hyperparameters of prior density based 
on the moment estimate is proposed..., Bayes estimation is used to estimate the 
parameters.) 

Huo and Blei are analogous art because Huo's paper concerns the training of the 
HMM model parameters that are used in Blei. It would be obvious to someone of 
ordinary skill in the art at the time of the invention to combine Huo with the Blei and 
Utiyama device because Huo provides algorithms that "are shown to be effective 
especially in the cases in which the training or adaptation data are limited" which would 
provide an improvement over previous algorithms. 

Claims 18 and 19 are rejected under the same principles for being the apparatus 
claims to the corresponding method claims 7 and 8. Each of the stated corresponding 
claims have parallel limitations between the method and the device and the hardware 
aspect of claims 1 8 and 1 9 are taught by (Blei, Page 348, column 2) where it would 
have been obvious to someone of ordinary skill in the art at the time of the invention that 
the SpeechBot system could have been a hardware computer system. 
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Conclusion 

16. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. Refer to PTO-892, Notice of References Cited for a listing of 
analogous art. 

17. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

18. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to GREG A. BORSETTI whose telephone number is 
(571)270-3885. The examiner can normally be reached on Monday - Thursday (8am - 
5pm Eastern Time). 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, RICHEMOND DORVIL can be reached on 571-272-7602. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 
273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/Greg A. Borsetti/ 
Examiner, Art Unit 2626 



4/24/2009 



/Talivaldis Ivars Smits/ 
Primary Examiner, Art Unit 2626 



